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Abstract 


Vision  researchers  base  their  work  on  assumptions  that  lenses  behave  ideally.  In  practice  the 
image  formation  process  is  complex  and  camera  lenses  have  many  non-ideal  behaviors  that 
can  cause  problems  in  simple  vision  tasks  like  color  image  analysis  and  range  from  focus. 
For  example,  unlike  the  ideal  lens  the  refractive  power  of  a  real  lens  is  actually  a  function 
of  wavelength.  This  means  that  images  of  a  scene  taken  under  different  wavelengths  will 
have  slightly  different  focus  and  magnification.  The  resulting  misregistration  between  the 
images  when  they  are  superimposed  to  form  a  composite  color  image  can  cause  significant 
problems  in  color  image  analysis.  Another  property  of  real  lenses  is  that  focusing  the  lens 
also  changes  its  magnification.  In  range  from  focus  these  changes  in  image  magnification 
can  bias  the  position  of  a  sharpness  criterion  function’s  peak  leading  to  errors  in  the  range 
estimates. 

By  precisely  controlling  the  lens  during  imaging  it  is  possible  to  make  lenses  behave 
more  ideally.  VVe  have  developed  a  procedure  we  call  .Active  Color  Imaging  in  which  lens 
magnification  and  focus  are  precisely  compensated  to  produce  color  images  that  are  over 
10  times  better  registered  than  uncompensated  color  images.  VVe  have  also  developed  a 
procedure  we  call  Constant  Magnification  Focusing  which  allows  a  user  to  focus  a  lens 
without  changing  the  image  magnification,  eliminating  focus  magnification  induced  errors 
in  range  from  focus  tasks. 


1  Introduction 


The  ideal  lens  for  machine  vision  would  probably  behave  like  a  pinhole  camera  -  with  perfect 
perspective  projection  i.jm  every  point  in  the  object  space  down  to  the  image  plane  and 
with  every  point  in  object  space  in  perfeci  focus.  Such  a  lens  is  not  practical,  first,  because 
it  would  admit  insufficient  light  for  most  imaging  sensors,  and  second,  because  diffraction 
effects  at  the  camera’s  pinhole  would  limit  the  image  resolution  that  could  be  obtained. 
In  real  machine  vision  svstems  the  pinholes  are  replaced  by  lenses  to  gather  light  from 
many  points  in  the  c’oject  space  and  project  it  down  to  the  imaging  plane.  Such  lenses 
e.xhibit  many  non-ideal  behaviors  that  result  from  the  fundamental  physics  involved  in  the 
optics  and  from  manufacturing  considerations.  For  the  most  part  vision  researchers  make 
simplifying  assumptions  about  the  image  formation  process  and  ignore  most  non-ideal  lens 
behaviors.  In  this  report  we  e.xamine  how  two  low  level  vision  tasks,  color  image  analysis 
and  focus  ranging,  are  affected  by  two  non-ideal  lens  behaviors.  Subsequently  for  both  cases 
we  show  how  precise  control  of  the  lens  during  imaging  can  compensate  for  the  non-ideal 
lens  behaviors  and  provide  significantly  better  results. 

In  color  image  analysis  the  information  contained  in  three  color  bands  is  used  to  de¬ 
termine  properties  of  the  scene  being  imaged.  Implicit  in  the  color  image  analysis  is  the 
assumption  that  the  per  pi.xel  information  in  each  band  corresponds  to  the  same  point, 
region,  or  volume  in  object  space.  In  this  report  we  show  how  chromatic  aberration  in 
lenses  can  result  in  significant  misregistration  between  the  bands  of  color  images,  and  how 
this  misregistration  can  cause  problems  for  color  image  analysis.  We  then  show  how  pre¬ 
cise  control  of  the  lens’  focal  length,  focus  distance  and  XY  position  during  imaging  can 
reduce  the  misregistration  between  the  color  image  bands  by  an  order  of  magnitude.  We 
call  this  method  .Active  Color  Imaging  (ACI).  .As  part  of  this  work  we  present  a  method  for 
measuring  the  amount  of  lateral  chromatic  aberration  in  a  camera  system. 

In  range  from  focus  the  distance  to  a  feature  in  the  camera’s  field  of  view  is  estimated  by 
moving  the  camera  lens  through  a  series  of  focus  positions  while  determining  the  sharpness 
of  the  feature’s  image  at  each  position.  The  feature’s  sharpness  is  measured  using  a  criterion 
function  evaluated  over  the  image.  Given  the  focus  position  of  the  lens  where  the  criterion 
function  peaks,  the  distance  between  the  camera  and  the  feature  in  the  image  can  be 
determined  using  a  calibrated  lens  model.  Implicit  in  the  range  from  focus  procedure  is 
the  assumption  that  the  criterion  function  peaks  at  the  point  of  sharpest  focus.  In  this 
report  we  show  how  focus  magnification  can  bias  the  position  of  the  criterion  function 
peak,  yielding  incorrect  results  in  the  focus  ranging  task.  We  then  show  how  nulling  out 
focus  magnification  by  the  precise  control  of  the  lens’  focal  length  can  eliminate  the  bias  in 
the  position  of  the  criterion  function’s  peak.  We  call  this  approach  Constant  Magnification 
Focusing  (CMF).  As  part  of  this  work  we  present  a  method  to  accurately  and  reliably 
estimate  the  position  of  unimodal  criterion  function  peaks  in  the  presence  of  noise. 

This  work  was  originally  presented  in  Willson  [17]  at  the  1991  IEEE  International  Con¬ 
ference  on  Robotics  and  .Automation. 
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2  Active  Color  Imaging 

Color  image  analysis  uses  the  information  contained  in  three  spectral  bands  to  determine 
properties  of  the  scene  being  imaged.  Implicit  in  any  color  image  analysis  is  the  assumption 
that  the  per  pixel  information  in  each  band  corresponds  to  the  same  point,  region,  or  volume 
in  object  space.  As  we  will  demonstrate  in  the  following  section,  this  is  not  always  so. 

The  starting  point  for  any  type  of  color  image  analysis  is  the  acquisition  of  color  images. 
To  simplify  the  image  alignment  problem  between  the  color  bands  virtually  all  color  imaging 
processes  use  a  single  lens  to  image  all  of  the  bands.  Variations  between  imaging  processes 
occur  in  the  methods  used  to  digitize  the  bands.  Three  basic  approaches  are  used.  The 
simplest  approach,  used  in  most  consumer  color  cameras,  is  to  use  a  single  sensor  with  every 
third  pixel  overlaid  with  one  of  the  three  bandpass  filters.  Spatial  interpolation  is  then  used 
fill  in  the  gaps  in  the  image  bands. 

The  second  approach,  used  in  broadcast  quality  color  cameras,  is  to  split  the  image  from 
the  lens  into  three  and  then  to  project  each  of  the  three  images  through  bandpass  filters 
onto  separate  sensors. 

The  third  approach,  used  in  still  color  imaging,  is  to  place  three  bandpass  filters  in  the 
optical  path  of  the  camera  system  one  after  another  and  take  the  three  images  with  the 
same  sensor.  This  third  approach  is  the  one  that  we  use  in  the  Calibrated  Imaging  Lab,  and 
is  used  for  most  machine  vision  research  in  color  physics  because  of  the  superior  registration 
between  images  and  the  improved  SNR  that  is  possible  for  each  band. 

The  imaging  system  used  for  this  work  consists  of  a  solid  state  video  camera,  a  computer 
controlled  filter  wheel,  a  computer  controlled  lens  and  a  computer  controlled  six  degree  of 
freedom  camera  platform.  The  camera  is  a  General  Imaging  MOS-5300  connected  to  a 
Matrox  frame  grabber.  The  filter  wheel  houses  six  filters  including  Wratten  #25  (red),  58 
(green)  and  47B  (blue).  The  lens  is  a  Cosmicar  CCTV  grade  zoom  lens  with  a  focal  length 
range  of  12.5  to  75mm,  a  minimum  focussed  distance  of  1.2m,  and  a  maximum  aperture 
ratio  of  1:1.8.  Lens  automation  is  provided  by  three  micro  stepping  motors  providing  3900 
steps  of  resolution  for  focus  distance,  4000  steps  for  focal  length  and  2700  steps  for  aperture. 
The  camera  platform  has  a  positioning  accuracy  of  0.025mm. 

2.1  Chromatic  Aberration 

Chromatic  aberration  exists  in  camera  lenses  because  the  index  of  refraction  of  optical 
components  varies  as  a  function  of  wavelength.  This  difference  in  refractive  index  causes  the 
different  wavelengths  of  light  to  be  refracted  or  bent  different  amounts  by  the  components 
of  the  lens.  For  example,  given  a  simple  uncorrected  thin  lens  with  incident  white  light  rays, 
the  blue  components  of  the  incident  rays  will  be  brought  to  a  focus  closer  to  the  lens  than 
the  red  components,  as  shown  in  Figure  1.  In  [5]  Funt  and  IIo  make  use  of  this  property  to 
extract  spectral  information  to  address  the  problem  of  color  constancy. 

Chromatic  aberration  is  an  intrinsic  property  of  a  camera  lens.  It  can  be  partially 
compensated  for  in  the  lens  optics  by  using  pairs  of  lens  elements  with  offsetting  dispersion 
factors.  Unfortunately  such  compensation  is  usually  done  for  only  two  wavelengths  (rea 
and  blue)  and  then  only  at  two  points  in  the  image  field:  at  the  optical  a.\js  and  at  some 
specified  radial  distance  from  the  optical  axis  [14].  For  a  more  detailed  discussion  of  lenses 
and  chromatic  aberration  the  reader  is  referred  to  [13]. 

To  measure  chromatic  aberration  we  use  black  on  white  checker  board  targets  like  the 
target  shown  in  Figure  2.  The  positions  of  the  vertical  and  horizontal  black  to  white  step 
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Figure  1:  Chromatic  aberration  in  a  thin  lens 


edges,  measured  in  each  of  the  bands,  provide  a  fairly  precise  measure  of  the  lateral  chro¬ 
matic  aberration  in  the  camera  system.  Appendix  A  describes  the  measurement  procedure 
in  detail. 

Effects  of  Chromatic  Aberration  on  Images 

Chromatic  aberration  has  three  effects  on  color  images:  magnification  differences  between 
bands,  focus  differences  between  bands,  and  decentering  between  the  bands  11  .  The  first 
two  of  these  effects  are  illustrated  in  Figure  1.  In  this  figure  the  difference  in  the  heights 
(magnification)  of  the  two  images  is  called  lateral  chromatic  aberration.  The  difference 
in  the  positions  of  the  plane  of  focus  for  the  two  images  is  called  longitudinal  chromatic 
aberration.  These  effects  can  also  be  seen  by  examining  the  color  image  of  the  black  on 
white  grid  shown  in  Figure  3.  Figure  4  is  a  graph  of  the  red,  green  and  blue  pixel  values 
versus  pixel  number  for  the  scanline  located  in  section  1  of  the  grid  image  in  Figure  3. 
The  vertical  black  line  of  the  grid  that  falls  within  section  1  is  near  the  optical  axis  of  the 
lens  and  shows  no  noticeable  misregistration  between  the  red,  green  and  blue  color  bands. 
Figure  5  is  a  graph  of  the  red.  green  and  blue  pixels  versus  pixel  number  for  section  2  of  the 
same  scanline.  The  vertical  black  line  being  examined  in  this  figure  is  relatively  far  from 
the  optical  axis  of  the  lens  and  clearly  shows  the  misregistration  of  the  three  color  bands 
due  to  lateral  chromatic  aberration.  The  slightly  broader  width  of  the  red  valley  in  both 
figures  shows  the  defocus  in  the  red  image  caused  by  longitudinal  chromatic  aberration  (the 
lens  was  initially  focussed  with  blue  light).  With  our  camera  system  the  difference  in  the 
magnification  between  the  uncorrected  red  and  blue  images  is  on  the  order  of  0.5%.  or  1.2 
pixels  near  the  outer  edges  of  images  that  are  512  pixels  wide  (ie.  a  1.2  pixel  displacement 
in  one  direction  at  the  left  edge  and  a  1.2  pixel  displacement  in  the  other  direction  at  the 
right  edge). 

While  magnification  and  defocus  result  from  intrinsic  properties  of  the  optical  compo¬ 
nents,  the  third  chromatic  effect,  decentering  between  the  image  bands,  is  likely  the  result 
of  misalignment  in  the  optical  components  of  the  lens.  This  causes  light  rays  of  different 
wavelengths  to  take  slightly  different  optical  paths  through  the  lens.  The  decentering  ef- 
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Figure  3:  Image  of  grid  pattern 
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feet  is  compounded  by  optical  roll  out,  where  changing  the  lens’  focussed  distance  and  focal 
length  settings  changes  the  degree  of  misalignment,  causing  a  drift  in  the  image  center.  This 
effect  has  also  been  noted  by  Lenz  [8],  Nair  [9]  and  Wiley  [16].  Whereas  the  magnification 
differences  are  radially  symmetric  with  respect  to  the  center  of  the  optical  system  and  have 
a  smaller  magnitude  nearer  the  center  than  at  the  edge,  decentering  differences  affect  the 
entire  image  uniformly.  In  our  system  the  measured  translation  between  the  uncorrected 
red,  green  and  blue  bands  ranges  from  0.1  to  0.4  pixel  widths,  depending  upon  the  image 
band  and  the  lens  settings. 

Effects  on  color  analysis 

The  demagnification,  defocus  and  decentering  between  the  color  image  bands  means  that 
the  three  band  values  for  each  pixel  do  not  in  fact  correspond  to  the  same  point  in  object 
space.  This  has  rather  obvious  impacts  on  color  image  analysis.  For  example,  regions  near 
rapid  color  changes  (eg.  near  highlights  or  the  edges  of  features),  will  show  hue  shifts.  If  we 
were  to  look  closely  at  the  pixels  in  section  2  of  Figure  3  (shown  in  profile  in  Figure  5).  going 
left  to  right,  we  would  see  the  pixel  colors  change  from  white,  to  bluish,  to  black,  to  reddish, 
and  back  to  white.  As  we  will  show  later,  these  hue  shifts  show  up  as  erroneous  points  in 
color  histograms  of  the  image  which  in  turn  can  lead  to  color  segmentation  problems  [6]. 
Edge  detection  and  edge  localization  in  color  images  are  also  confounded  in  obvious  ways. 

2.2  Compensation  Approach 

The  most  direct  approach  to  compensating  for  chromatic  aberration  is  to  modify  the  magni¬ 
fication  and  focussed  distance  for  each  of  the  color  bands  to  nuU  out  the  magnification  and 
defocus  introduced  by  the  chromatic  aberration.  Correction  for  image  decentoring  can  be 
accomplished  by  shifting  the  camera  in  the  A'F  (sensor)  plane.  We  call  this  compensation 
approach  .\ctive  Color  Imaging. 

In  our  present  color  imaging  system  the  test  target  shown  in  Figure  6  is  first  placed 
directly  in  front  of  the  scene  to  be  imaged  so  that  the  compensated  lens  settings  can  be 
determined.  The  black  to  white  edges  on  the  target  are  used  to  focus  on  and  to  determine 
the  relative  image  magnification  and  shifting  between  the  color  bands.  Once  the  correct 
focus,  zoom  and  shift  settings  for  the  lens  have  been  determined  the  calibration  target  is 
removed  and  a  compensated  color  image  of  the  scene  is  taken. 

In  our  approach  the  blue  image  is  used  as  ground  truth.  In  the  calibration  procedure  the 
lens  settings  for  the  red  and  the  green  bands  are  determined  such  that  the  errors  between 
the  blue  and  red  images  and  the  blue  and  green  images  are  minimized.  To  determine  the 
compensated  lens  settings  for  the  red  and  green  images  we  first  find  the  position  of  best 
focus  for  each  of  the  three  bands  using  the  automated  focusing  algorithm  developed  by 
Krotkov  [7].  .Text  we  take  three  focus  corrected  images  and  determine  the  magnification 
differences  between  the  blue  and  red  and  the  blue  and  green  bands.  Using  a  calibrated  lens 
model  (described  in  appendix  B)  that  relates  the  image  magnification  to  the  positions  of  the 
focal  length  (zoom)  and  focussed  distance  motors,  we  then  change  the  zoom  appropriately 
for  the  red  and  the  green  bands  to  correct  the  magnification  differences  that  result  from 
the  chromatic  aberration  and  from  the  refocusing  operation.  Finally,  to  correct  the  color 
dependent  image  translation,  we  take  three  focus  and  magnification  corrected  images  and 
then  determine  the  amount  of  camera  shifting  that  is  needed  in  the  red  and  green  bands  to 
compensate  for  translation  introdticed  by  the  optics. 
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Calibration  example 


By  examining  the  positions  of  the  12  edges  on  a  scanline  crossing  the  center  of  the  calibration 
target  in  Figure  6  we  can  graphically  show  the  effects  of  each  of  the  steps  of  the  compensation 
procedure.  Plotting  the  difference  in  the  edge  positions  (in  pixel  widths)  betw'een  the 
uncorrected  blue  and  red  images  versus  the  edge  number,  we  obtain  the  solid  line  in  Figure  7. 
For  a  lens  with  no  chromatic  aberration  the  line  would  be  zero  everywhere,  indicating  no 
difference  in  the  positions  of  the  edges  in  the  blue  and  red  images.  Our  plot  shows  two 
effects.  The  slope  of  the  blue-red  line  is  a  measure  of  the  relative  magnification  of  the 
blue  and  red  images.  The  displacement  of  the  line  up  or  down  is  a  measure  of  the  X 
component  of  the  image  translation.  In  this  example  the  magnification  difference  between 
the  uncompensated  blue  and  red  images  is  -0.45%  and  between  the  blue  and  green  images 
is  -0.14%  ‘.  The  X  shift  between  the  blue  and  red  images  is  0.12  pixels  and  between  the 
blue  and  green  images  is  0.08  pi.xcls. 

The  doited  line  shows  the  blue- red  differences  after  refocusing  the  red  image.  The 
magnification  difference  has  now  shifted  in  the  other  direction  as  a  result  of  the  magnification 
change  introduced  by  refocusing.  After  focus  compensation  the  magnification  difference 
measured  between  the  blue  and  red  images  is  1.01%  and  between  the  blue  and  green  images 
is  0.35%.  The  .V  shift  between  the  blue  and  red  images  is  0.35  pixels  and  between  the  blue 
and  green  images  is  0.14  pi.xels. 

The  dashed  line  shows  the  blue-red  differences  after  rezooming  the  red  image.  While 
the  slope  of  the  line  is  approximately  zero,  indicating  that  the  magnification  differences 
between  the  bands  have  been  ebminated,  the  image  translation  is  now  obvious.  After  focus 
and  zoom  compensating  the  images  the  magnification  difference  between  the  blue  and  red 
images  is  0.07%  and  between  the  blue  and  green  images  is  0.01%.  The  A'  shift  between  the 
blue  and  red  images  is  0.46  pixels  and  between  the  green  and  blue  images  is  0.12  pi.xels. 

The  dash-dot  bne  shows  the  final  result,  after  the  camera  has  been  shifted.  The  plot 
is  now  close  to  zero  everywhere,  emulating  ideal  lens  behavior.  .\t  the  edge  of  the  image 
the  misregistration  has  dropped  from  about  one  pi.xel  to  about  one-tenth  of  a  pixel.  In  the 
final  focus/zoom/shift  compensated  images  the  magnification  difference  between  the  blue 
and  red  images  is  0.05%  and  between  the  blue  and  green  images  is  -0.01%.  The  X  shift 
between  the  blue  and  red  images  is  0.00  pi.xels  and  between  the  blue  and  green  images  is 
0.01  pi.xels. 
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Figure  6:  Active  Color  Imaging  calibration  target 
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Active  Color  Imaging  requires  very  fine  control  of  the  lens  parameters.  For  this  example 
the  lens  and  camera  parameters  were  changed  by  the  amounts  shown  in  the  following  table. 


band 

focus 

motor 

(out  of  3900) 

difference 
from  blue 
setting 

focal  length 
motor 

(out  of  4000) 

difference 
from  blue 
setting 

camera 

X  shift 

red 

1407 

-6.36% 

533 

+0.83% 

0.006  in 

green 

1592 

-1.62% 

508 

+0.20% 

0.002  in 

blue 

1655 

500 

In  this  example  the  focal  length  motor  was  initially  set  to  500  units  (//  «  67mm)  when 
the  camera  was  set  up.  At  the  start  of  the  Active  Color  Imaging  procedure  the  lens  was 
focussed  with  the  blue,  green  and  red  filters.  The  final  focus  motor  positions  are  listed  in 
column  2,  with  the  percentage  deviation  from  the  blue  settings  listed  in  column  3.  Using 
the  refocussed  images,  the  compensating  focal  length  motor  settings  were  calculated  for 
the  green  and  red  images.  The  final  zoom  motor  settings  are  Listed  in  cc.amn  4,  with  the 
percentage  deviation  from  the  blue  settings  in  column  5.  Finally,  using  the  focus  and  zoom 
compensated  images,  the  required  A'  camera  shift  was  calculated  for  the  green  and  red 
images.  These  values  are  shown  in  column  6.  Shifting  in  the  Y  direction  was  not  performed 
for  this  example. 

2.3  Experimental  Results 

To  measure  the  effects  of  active  lens  compensation  across  the  full  field  of  view  of  the  camera 
we  use  a  checkerboard  target  and  the  procedure  described  in  appendix  A.  Figures  8  and  9 
show  the  magnitude  of  the  image  misregistration  (in  pixel  widths)  between  the  blue  and  red 
images  coded  in  a  gray  scale  ranging  from  0  pixels  (black)  to  1.5  pixels  (white).  Figure  8 
shows  the  blue-red  misregistration  for  the  target  imaged  without  lens  compensation.  The 
misregistration  across  the  image  ranges  from  0  to  1.2  pixel  widths.  For  the  blue-red  case 
the  zero  error  region  appears  to  the  lower  right  of  the  center  of  the  image.  In  general  the 
location  of  the  zero  error  region  for  blue-green  case  will  not  be  the  same.  Figure  9  shows  the 
magnitude  of  the  blue-red  misregistration  for  the  target  imaged  with  lens  compensation. 
The  remaining  misregistration  is  now  less  than  0.1  pixel  widths  over  most  of  the  image. 
So  with  active  lens  compensation  it  is  possible  to  reduce  the  image  band  misregistration 
introduced  by  chromatic  aberration  by  over  an  order  of  magnitude. 

.Another  way  of  showing  the  effects  of  active  lens  compensation  is  to  plot  image  pi.xels 
in  three  dimensional  RGB  space.  Figures  10  and  11  contain  color  histograms  of  the  region 
containing  the  rightmost  vertical  line  of  the  grid  image  shown  in  Figure  3.  Figure  3  is  a 
color  image  of  a  black  grid  on  a  white  background  so  ideally  all  of  the  pixels  in  the  image 
should  be  black,  white,  or  shades  of  gray.  The  plotted  pixels  from  the  region  containing 
the  grid  line  should  form  a  straight  line  of  grays  between  the  lower  left  corner  of  the  color 
cube  (corresponding  to  black)  and  the  upper  right  corner  of  the  cube  (corresponding  to 
white).  This  is  not  the  ca.se  in  the  uncorrccted  image.  In  Figure  10  the  histogram  of  the 
pixels  from  the  uncorrected  image  shows  two  distinct  paths  from  black  to  white.  The  first 
path  bends  towards  the  red  corner  of  the  color  cube  while  the  second  path  bends  towards 
the  blue  corner  of  the  color  cube.  These  deviations  would  pose  a  serious  problem  for  any 
algorithm  trying  to  segment  the  image  by  using  regions  in  the  color  space.  In  Figure  11  the 
histogram  of  the  pi.xels  from  the  image  taken  with  active  lens  compensation  shows  a  much 
tighter  grouping  of  pixel  values  along  the  gray  line. 
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Figure  10;  Color  histogram  -  ur 


2.4  Conclusions 


With  the  lenses  typically  used  in  computer  vision^,  the  processes  by  which  color  images 
are  taken  will  generally  produce  images  that  have  magnification,  focus  and  image  center 
differences  between  the  color  bands.  These  effects  are  due  both  to  chromatic  aberration 
and  to  optical  misalignment  in  the  camera  lens.  The  end  result  is  significant  levels  of 
misregistration  and  defocus  in  the  final  composite  image.  With  active  compensation  of  the 
lens  settings  and  the  camera  position  the  misregistration  between  the  color  bands  of  an 
image  can  be  reduced  by  an  order  of  magnitude. 

One  alternative  to  active  lens  control  for  color  image  correction  is  to  post  process  the 
image  bands  to  remove  the  chromatic  aberration  effects.  In  [2]  Boult  compares  our  active 
lens  control  approach  to  their  image  warping  approach  for  chromatic  aberration  correction. 
The  major  advantages  of  image  warping  are  that  it  can  be  applied  to  cameras  that  take 
RGB  images  in  parallel  (eg.  color  video  cameras),  it  does  not  require  specird  lens  control 
hardware,  and  it  can  handle  chromatically  varying  geometric  distortions  that  cannot  be 
corrected  by  focusing  or  zooming.  The  disadvantages  of  the  image  warping  approach  are 
its  inability  to  compensate  for  the  defocus  between  image  bands  and  the  relatively  large 
amount  of  calibration  data  that  it  can  require.  Boult  found  that  while  image  warping  com¬ 
pared  reasonably  well  both  qualitatively  and  quaiitatively  with  the  active  lens  compensation 
approach,  the  active  lens  compensation  approach  could  produce  better  overall  results  for 
simple  uncorrected  lenses  because  of  its  ability  correct  for  focus  differences  between  bands. 

In  our  present  color  imaging  procedure  we  determine  the  control  settings  for  the  com¬ 
pensated  images  by  imaging  a  test  target  to  measure  the  amounts  of  defocus,  magnification 
change  and  decentering  between  the  color  bands.  We  are  now  proceeding  to  develop  cal¬ 
ibrated  lens  models  that  relate  the  focussed  distance,  magnification  and  optical  center  of 
the  lens  to  the  camera’s  control  motors  and  the  imaging  color.  With  these  models  it  will 
be  possible,  given  the  lens  settings  for  one  color  band,  to  calculate  the  settings  for  the 
remaining  two  bands  without  the  necessity  of  a  test  target. 


^Wp  have  mea.snrp(]  comparable  levels  of  chromatic  aberration  in  a  wide  variety  of  lenses,  including 
CCTV  lenses.  3.5mm  SLR  lenses  and  ENG/EFP  color  TV  lenses,  including  lenses  that  are  adverti.sed  as 
being  corrected  to  eliminate  chromatic  aberration. 
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focus  positions 


Figure  12:  Focus  magnification 


3  Constant  Magnification  Focusing 

In  range  from  focus  the  distance  to  a  target  is  estimated  by  moving  the  camera  lens  through 
a  series  of  focus  positions  while  determining  the  sharpness  of  the  target's  image  at  each 
position.  The  target's  sharpness  is  determined  by  evaluating  a  sharpness  criterion  function 
over  the  image.  Given  the  focus  position  of  the  lens  where  the  criterion  function  peaks, 
the  distance  between  the  camera  and  the  target  can  be  determined  using  a  calibrated  lens 
model.  Implicit  in  the  range  from  focus  procedure  is  the  assumption  that  the  criterion 
function  peaks  at  the  point  of  sharpest  focus^.  There  is  also  the  implicit  assumption  that 
the  value  of  the  criterion  function  varies  only  with  the  focussed  distance  of  the  lens.  In  the 
following  section  we  demonstrate  how,  for  the  most  widely  used  class  of  criterion  functions, 
the  criterion  function  value  (and  the  position  of  its  peak)  is  also  dependent  on  the  lens' 
aperture  and  on  the  content  of  the  image. 

-Manv  sharpness  criterion  functions  have  been  suggested  for  focus  ranging  7  .  In  our 
experiments  we  make  use  of  a  sum  of  squared  image  gradients  function,  also  called  the 
Tenengrad  function.  Most  of  our  discussion  transcends  a  specific  choice  of  criterion  function. 

3.1  Focus  Magnification 

For  a  fixed  focal  length  lens,  focus  magnification  is  the  change  in  image  magnification  that 
results  as  the  camera's  sensing  plane  is  moved  along  the  optical  axis  in  order  to  vary  the 
lens'  focussed  distance.  This  is  illustrated  in  Figure  12.  Effectively  focus  magnification 
causes  the  image  to  scale  up  as  the  lens  is  focussed  from  far  to  near.  For  a  variable  focal 
length  (^oom)  lens  focus  magnification  is  conceptually  the  same. 

If  one  considers  focus  magnification  then  the  net  magnification  of  a  zoom  lens  is  actually 
a  fuuition  of  two  lens  variables:  the  focal  length  and  the  focussed  distance.  By  measuring 
the  dimensions  of  a  target  across  a  range  of  lens  settings  we  have  produced  a  calibrated 
lens  model  relating  the  focal  length  and  focussed  distance  motor  settings  to  the  relative 

'The  point  of  best  focus  is  usually  defined  as  the  point  at  which  the  region  of  interest  has  the  highest 
spatial  frequency  content 
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Figure  13:  Relative  magnification  versus  f,i  and  //  motor  positions 

image  magnification  {M  =  g(fhfd))-  Appendix  B  describes  the  relative  magnification 
model  in  detail.  Figure  13  shows  a  plot  of  the  relative  image  magnification  versus  the 
focussed  distance  motor  position  and  focal  length  motor  position  for  the  lens  used  in  these 
experiments. 

Image  effects 

Focus  magnification  causes  three  significant  problems  for  range  from  focus:  feature  clipping, 
perimeter  scaling,  and  gradient  scaling.  The  first  of  these  effects,  illustrated  in  Figure  14. 
iavoives  tiie  movement  of  features  into  or  out  of  the  window  of  interest  as  the  image  mag¬ 
nification  changes.  This  effect  is  pretty  straight  forward  and  has  been  noted  by  several 
researchers,  including  Krotkov  [7].  Darrell  [4],  .\air  [9]  and  Nayar  [10].  The  second  effect, 
perimeter  scaling,  illustrated  in  Figure  1.5.  involves  the  proportional  scaling  of  the  length  of 
feature  perimeters  as  the  image  magnification  is  changed.  Effectively  this  is  image  scaling 
done  perpendicular  to  the  direction  of  image  intensity  gradients.  The  final  effect,  gradient 
scaling,  illustrated  in  Figure  10,  involves  the  inverse  scaling  of  the  width  a.  d  slope  of  inten- 
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Figure  14:  Focus  magnification  -  feature  clipping 


sity  gradients  as  the  magnification  changes.  Effectively  this  is  image  scaling  done  parallel 
to  the  direction  of  the  image  intensity  gradients. 


Effects  on  focus  ranging 

For  accurate  focus  ranging  we  would  like  the  position  of  the  peak  of  the  criterion  func¬ 
tion  to  depend  solely  on  the  focussed  distance  of  the  lens.  .A.s  we  will  demonstrate,  focus 
magnification  can  make  the  position  of  the  peak  sensitive  to  additional  factors. 

The  first  focus  magnification  effect,  feature  clipping,  causes  abrupt  changes  in  the  crite¬ 
rion  function  value  as  features  leave  or  enter  the  evaluation  window.  These  changes  in  turn 
mav  result  in  multiple  or  false  criterion  function  peaks  which  can  lead  to  erroneous  range 
estimates.  The  last  two  focus  magnification  effects,  perimeter  scaling  and  gradient  scaling, 
cause  more  subtle  biases  in  the  position  of  the  criterion  function  peak. 

With  focus  magnification  the  size  of  an  image  scales  up  as  the  lens  is  focussed  from  far 
to  near.  .\s  the  image  scales  up  feature  perimeters  will  also  be  proportionately  scaled  up 
causing  the  value  of  the  criterion  function  to  increase.  This  increase  in  turn  causes  a  bias 
in  the  position  of  the  criterion  function  peak,  making  the  point  of  best  focus  appear  too 
near.  However,  as  the  image  scales  up  with  focus  magnification,  the  intensity  gradients  on 
features  will  also  spread  out  laterally.  If  the  width  b  of  the  gradient  shown  in  Figure  16  is 
significantly  larger  than  the  width  of  the  point  spread  function  of  the  focussed  lens,  then  the 
blur  introduced  by  the  defocusing  lens  will  not  significantly  alter  the  value  of  the  gradient 
across  most  of  the  width  b.  In  the  one  dimensional  case  for  a  sum  of  squared  gradients 
criterion  function,  the  value  of  the  criterion  function  is  given  by 


value  = 


a 


2 


b  ■ 


Thus  as  the  image  (and  6)  are  scaled  up.  the  value  of  the  criterion  function  is  proportionately 
scaled  down.  This  decrease  causes  a  bias  in  the  position  of  the  criterion  function  peak 
making  the  point  of  best  focus  appear  too  far  away. 

Both  the  perimeter  and  gradient  scaling  effects  are  functions  of  the  image  magnification 
and  the  rate  of  blurring  as  the  lens  is  moved  away  from  the  point  of  best  focus.  The  blur 
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Figure  15:  Focus  magnification  -  perimeter  scaling 
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Figure  16:  Focus  magnification  -  gradient  scaling 
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Figure  17:  Box  and  bar  focus  targets 

rate  is  effectively  a  function  of  the  lens'  aperture.  The  net  result  of  the  two  competing 
effects  is  an  additive  bias  to  the  position  of  the  criterion  function  peak  which  is  dependent 
on  the  lens’  aperture  and  on  the  content  of  the  image. 

3.2  Compensation  Approach 

To  avoid  focus  magnification  effects  in  focus  ranging  researchers  have  suggested  either  keep¬ 
ing  the  image  magnification  constant  ’4'  or  keeping  the  evaluation  window  large  enough  to 
completely  encompass  the  features  being  ranged  7’.  As  suggested  above  however,  simply 
keeping  the  features  being  ranged  entirely  within  the  evaluation  window  will  not  overcome 
all  of  the  focus  magnification  effects.  Only  a  constant  image  magnification  approach  can 
avoid  the  problems  that  result  from  focus  magnification. 

In  4  Darrell  suggests  two  approaches  for  dealing  with  focus  magnification.  The  first 
approach,  and  the  one  implemented  by  Darrell,  is  to  cancel  out  the  magnification  change 
by  scaling  the  images  in  software  after  they  are  taken.  This  approach  is  relatively  simple 
and  works  even  for  fixed  focal  length  lenses.  One  major  drawback  with  this  approach  is 
that  as  the  image  is  scaled,  the  spatial  interpolation  and  resampling  process  will  smooth 
the  image  and  influence  the  value  of  the  criterion  function.  .4n  additional  drawback  is  the 
computational  effort  required  to  scale  the  images. 

The  second  approach  suggested  by  Darrell  is  to  null  out  the  focus  magnification  by 
changing  the  lens'  focal  length  before  the  images  are  taken.  While  being  more  efficient  and 
effective  than  scaling  images,  this  approach  requires  precise  active  lens  control  as  the  focus 
distance  of  the  lens  is  varied.  For  our  system,  given  the  current  focussed  distance  and  focal 
length  settings  and  a  new  focussed  distance  setting,  we  use  the  relative  magnification  model 
described  in  appendix  B  to  find  a  new  focal  length  setting  which  maintains  the  same  relative 
magnification.  We  call  this  compensation  approach  Constant  Magnification  Focusing. 

3.3  Experimental  Results 

To  illustrate  the  effects  of  focus  magnification  on  the  position  of  the  Tenengrad  criterion 
function  peak  we  use  two  targets,  illustrated  in  Figure  17.  Target  1  is  a  black  square  on  a 
white  background,  completely  enclosed  in  the  evaluation  window.  Target  2  is  a  black  bar 
on  a  white  background  with  only  the  center  region  of  the  bar  contained  in  the  evaluation 
window.  For  both  targets,  the  majority  of  the  value  of  the  criterion  fimction  results  from 
the  black  to  white  edge,  with  the  totally  white  and  totally  black  regions  contributing  in¬ 
significant  amounts.  For  target  1  the  length  of  the  perimeter  of  the  black  region  changes 
with  the  focus  magnification,  while  for  target  2  the  perimeter  length  remains  constant.  For 
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both  targets  the  width  of  the  gradient  at  the  black  to  white  edge  is  not  large  enough  (rela¬ 
tive  to  the  lens’  point  spread  function)  to  produce  gradient  scaling  effects.  The  box  and  bar 
targets  are  both  located  on  the  same  plane  exactly  1.5m  ±5mm  away  from  the  camera’s 
sensor  plane. 

For  these  experiments  we  use  a  sum  of  squared  image  gradients  criterion  function  with 
the  image  gradients  being  calculated  using  a  3x3  Sobel  operator.  The  position  of  the 
criterion  function  peaks  are  determined  by  least  squares  fitting  the  data  to  a  quadratic 
curve,  and  then  solving  the  quadratic  for  the  position  of  the  peak.  Appendix  C  describes 
the  peak  locating  procedure  in  detail. 

To  show  the  effects  of  focus  magnification  two  different  blur  rates  are  selected  by  using 
two  different  aperture  values.  At  higher  blur  rates  (wider  apertures,  narrower  depths  of 
field)  the  value  of  the  Tenengrad  function  falls  off  too  quickly  for  focus  magnification  effects 
to  significantly  bias  the  position  of  the  criterion  function  peak.  At  lower  blur  rates  (narrower 
apertures,  wider  depths  of  field)  the  value  of  the  Tenengrad  function  falls  off  more  slowly 
and  focus  magnification  has  a  more  pronounced  effect. 

The  experiments  are  done  first  without  and  then  with  Constant  Magnification  Focusing. 
For  the  uncompensated  focusing  the  focussed  distance  motor  is  varied  while  the  focal  length 
motor  is  held  constant  at  500  motor  units  (//  =5  67mm).  The  image  magnification  over  the 
range  of  focus  changes  by  a  factor  of  0.816.  For  Constant  Magnification  Focusing  the 
focussed  distance  motor  is  varied  while  the  focal  length  motor  is  concurrently  varied  to 
keep  the  magnification  constant.  Figure  18  shows  the  focal  length  motor  settings  versus  the 
focussed  distance  motor  settings  for  both  the  compensated  and  the  uncompensated  focusing. 
For  compensated  focusing  the  focal  length  motor  settings  are  varied  from  747  motor  units 
(fi  63mm)  to  277  motor  units  (//  ss  71mm).  The  Constant  Magnification  Focusing  curve 
is  essentially  an  isomagnification  contour  from  the  magnification  model  shown  in  Figure  13. 

Figure  19  shows  that  for  a  wider  aperture  the  positions  of  the  criterion  function  peaks  are 
the  same  for  both  the  box  and  bar  targets,  both  with  and  without  magnification  compensa¬ 
tion.  In  contrast  Figure  20  shows  that  for  a  narrower  aperture  with  no  compensation  there 
is  a  significant  bias  in  the  peak  position  for  the  box  target  towards  near  focus,  as  predicted 
earlier.  When  the  focus  magnification  is  compensated  for,  the  bias  in  the  peak  position  for 
the  box  target  is  eliminated.  With  an  actual  range  of  1.5m,  the  bias  in  the  uncompensated 
box  target’s  peak  position  corresponds  to  a  6%  error  in  the  range  measurement  determined 
from  the  calibrated  lens  model. 

Even  without  focus  magnification  effects  the  position  of  the  criterion  function  peak  varies 
with  the  size  of  the  aperture.  This  can  be  seen  more  clearly  in  Figure  21  which  shows  similar 
variations  in  the  peak  positions  for  a  range  of  aperture  positions  for  both  the  box  and  bar 
targets.  The  error  bars  in  this  graph  are  the  estimated  standard  deviations  of  the  criterion 
function  peak  position,  ais  calculated  by  the  peak  finding  procedure  described  in  appendix 
C.  One  potential  explanation  for  this  variation  is  spherical  aberration  in  the  optics.  .\s  the 
aperture  is  opened  the  addition  of  marginal  rays  to  the  image  changes  the  distribution  of 
light  in  the  3  dimensional  point  spread  function  of  the  lens,  potentially  altering  the  position 
of  the  point  spread  function’s  centroid. 
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Figure  18:  Lens  settings  for  range  from  focus 
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3.4  Conclusions 


In  range  from  focus  tasks  where  the  lens’  focus  is  varied  without  magnification  compensation 
the  position  of  the  sharpness  criterion  function’s  peak  can  potentially  be  bicised  away  from 
the  point  of  best  focus  by  several  image  scaling  effects.  The  amount  of  bias  depends  on 
the  size  of  the  aperture  and  on  the  content  of  the  image  region  being  evaluated.  With 
active  lens  compensation,  focus  magnification  induced  bias  problems  can  be  efficiently  and 
effectively  eliminated. 

Our  experiments  have  shown  that  the  position  of  criterion  function  peaks  also  varies  with 
lens  aperture,  possibly  due  to  spherical  aberration  within  the  lens.  Whatever  the  cause  this 
behavior  implies  that  a  calibrated  lens  model  that  relates  lens  settings  to  focussed  range 
will  have  to  include  a  parameter  for  the  aperture  position. 

Our  work  with  chromatic  aberration  has  shown  the  strong  dependence  of  a  lens’  focussed 
distance  on  the  wavelength  of  the  light  involved.  This  suggests  that  to  eliminate  chromatic 
aberration  as  a  source  of  error  in  range  from  focus,  all  lens  model  calibration  and  range 
from  focus  measurements  should  all  be  done  from  within  a  single  color  band. 
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4  Summary 


In  this  report  we  have  demonstrated  how  non-ideal  lens  behaviors  can  affect  low  level  imag¬ 
ing  tasks  and  have  introduced  precise  active  lens  compensation  as  a  way  to  overcome  some 
of  the  resulting  problems.  For  color  image  analysis  we  have  shown  how  chromatic  aber¬ 
ration  can  cause  image  band  misregistration  and  defocus,  which  in  turn,  can  significantly 
degrade  the  performance  of  color  image  analysis  algorithms.  To  compensate  for  chromatic 
aberration  we  have  developed  a  new  color  imaging  procedure  we  call  Active  Color  Imaging 
which  reduces  the  inter  band  misregistration  in  color  images  by  an  order  of  magnitude.  We 
have  also  developed  an  approach  to  measuring  lateral  chromatic  aberration  across  the  full 
field  of  view  of  a  camera  system. 

For  focus  ranging  we  have  shown  how  focus  magnification  can  cause  bicis  in  the  position 
of  criterion  function  peaks,  which  in  turn,  causes  incorrect  estimates  of  range.  To  compen¬ 
sate  for  focus  magnification  we  have  implemented  a  method  (first  proposed  by  Darrell),  that 
we  call  Constant  Magnification  Focusing  to  eliminate  the  effects  of  image  scaling  directly. 
.4s  part  of  this  work  we  have  also  developed  an  approach  to  accurately  and  reliably  estimate 
the  position  of  unimodaJ  criterion  function  peaks  in  the  presence  of  noise. 
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6  Appendix  A  -  Chromatic  aberration  measurement 

Our  basic  approach  to  measuring  lateral  chromatic  aberration  involves  accurately  measuring 
the  positions  of  black  to  white  step  edges  in  each  of  the  bands  of  a  color  image.  The  relative 
positions  of  the  edges  in  each  band  provide  a  measure  of  the  chromatic  aberration  induced 
misregistration  at  that  position  in  the  image.  By  using  a  dense  set  of  black  to  white  edges 
(eg.  a  checkerboard  pattern)  we  can  approximate  the  misregistration  caused  by  lateral 
chromatic  aberration  at  any  point  in  the  image  by  using  the  nearest  pairs  of  horizontal  and 
vertical  step  edges  to  determine  the  orthogonal  components  of  the  misregistration. 

Our  measurement  procedure  involves  two  steps.  In  the  first  step  one  band  of  a  checker¬ 
board  image  is  used  to  determine  a  set  of  reference  edges  who’s  positions  can  accurately  and 
reliably  be  measured.  In  the  second  step  the  positions  of  the  reference  edges  are  calculated 
to  subpixel  accuracy  in  each  of  the  image  bands. 

Reference  edge  selection 

Reference  edge  selection  begins  with  one  band  of  the  checkerboard  image,  shown  in  Fig¬ 
ure  22.  The  first  part  of  this  step  is  to  perform  thresholded  edge  detection  using  a  3x3  sobel 
operator.  The  resulting  image  is  shown  in  Figure  23.  Following  the  edge  detection  suitable 
reference  edges  are  selected  from  the  image.  Reference  edges  must  meet  four  criteria.  First, 
the  edges  must  be  fairly  close  to  horizontal  or  vertical.  Second,  the  edges  should  have  suf¬ 
ficient  length  so  that  any  shifting  component  in  the  direction  parallel  to  the  edge  wiU  stiU 
leave  enough  of  the  edge  to  measure  the  cross  edge  shifting  component.  Third,  the  edges 
should  be  sharp  enough  for  the  edge  position  to  be  accurately  calculated.  And  finally,  all 
of  the  horizontal  and  all  of  the  vertical  reference  edges  should  be  chosen  far  enough  apart 
so  that  there  is  no  ambiguity  in  tracking  them  between  image  bands. 

To  pick  out  reference  edges  we  test  the  edge  image  with  an  21.x21  pixel  test  mask. 
The  test  mask  for  finding  horizontal  edges  (used  for  detecting  vertical  shifting),  shown  in 
Figure  24,  contains  an  edge  area  and  two  non  edge  areas.  If  the  region  of  the  image  under  the 
test  mask  contains  an  edge  along  the  mask’s  edge  area,  and  no  edge  in  the  mask’s  non  edge 
area,  and  if  the  mask  does  not  overlap  any  previously  successful  masks,  then  the  section  of 
the  edge  under  the  mask  is  labeled  as  a  reference  edge.  Given  a  reference  edge,  the  image 
column  (or  row)  running  across  the  center  of  the  edge  is  used  to  measure  the  position  of 
the  edge  to  sub-pi.xel  accuracy  in  the  three  image  bands.  Figure  25  shows  the  edge  image 
with  the  vertical  and  horizontal  test  masks  superimposed.  The  tick  marks  across  the  edges 
identify  the  columns  and  rows  used  to  measure  the  positions  of  the  edges.  In  this  image 
there  are  415  vertical  reference  edges  and  275  horizontal  reference  edges. 

Edge  position  measurement 

The  second  step  in  measuring  lateral  chromatic  aberration  is  to  accurately  determine  the 
position  of  each  of  the  reference  edges  identified  in  the  reference  edge  selection  step.  To 
accomplish  this  we  use  a  standard  Laplacian-of-Gaussian  edge  localization  technique.  The 
LoG  technique  is  nearly  optimal  for  edge  localization  [3],  is  relatively  easy  to  implement  and 
is  computationally  very  efficient.  To  locate  the  position  of  an  edge  a  21  pi.xel  segment  of 
the  row  or  column  crossing  the  reference  edge  is  first  convolved  with  the  second  derivative 
of  a  Gaussian  convolution  kernel"*.  The  location  of  the  convolution's  zero  crossing  is  then 


^We  use  a  standard  deviation  of  4.0  pixels  for  the  (iaussian  convolution  kernel. 


Figure  22;  Checkerboard  image 


Figure  23:  Thresholded  edge  image 
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Figure  25:  Edge  image  with  highlighted  reference  edges 
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interpolated  to  give  the  exact  position  of  the  edge. 

Display  of  chromatic  aberration 

To  display  the  misregistration  due  to  lateral  chromatic  aberration  we  plot  the  difference 
between  the  positions  of  the  edges  in  the  blue  and  red  images  and  the  blue  and  green  images. 
This  can  be  done  for  both  the  vertical  edges  and  the  horizontal  edges.  If  the  vertical  and 
horizontal  reference  edges  are  close  enough  we  can  approximate  the  total  magnitude  of  the 
misregistration  at  any  point  in  the  image  by  using  the  misregistration  at  nearest  pairs  of 
vertical  and  horizontal  reference  edges  as  the  orthogonal  components. 

Figures  26  and  27  show  the  vertical  misregistration  between  the  blue  and  red  and  the 
blue  and  green  images  respectively.  Figures  28  and  29  show  the  horizontal  misregistra¬ 
tion  between  the  blue  and  red  and  the  blue  and  green  images  respectively.  Finally,  Fig¬ 
ures  .30  and  31  show  the  total  misregistration  between  the  blue  and  red  and  the  blue  and 
green  images  respectively. 


Figure  26:  Blue- Red  vertical  misregistration 


Figure  27:  Blue-Green  vertical  misregistration 
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Figure  28:  Blue-Red  horizontal  misregistration 


Figure  29:  Blue-Green  horizontal  misregistration 
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Figure  30:  Blue-Red  total  misregistration 


Figure  31:  Blue-Green  total  misregistration 
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7  Appendix  B  -  Relative  image  magnification  model 

Image  magnification  is  primarily  a  function  of  the  focal  length  of  a  lens.  However,  varying 
the  focussed  distance  of  the  lens  will  also  cause  changes  in  the  image  magnification.  While 
the  change  in  image  magnification  introduced  by  varying  the  focus  distance  is  somewhat 
smaller  than  the  change  introduced  by  varying  the  focal  length,  focus  induced  magnification 
changes  are  still  significant  in  color  imaging  tasks  and  in  range  from  focus  tasks.  For  these 
tasks  we  often  need  to  know  how  the  image  magnification  changes  in  response  to  changes  in 
both  the  lens’  focal  length  and  focus  distance  motors.  We  call  this  relationship  the  relative 
image  magnification  model  for  the  lens.  The  model  can  easily  be  extended  to  include  relative 
magnification  differences  that  result  from  imaging  in  different  color  bands. 

In  this  report  we  use  the  relative  magnification  model  primarily  for  magnification  equal¬ 
ization  across  two  or  more  images.  In  .\ctive  Color  Imaging  the  model  is  used  to  determine 
the  changes  in  the  focal  length  required  to  compensate  for  the  change  in  magnification 
introduced  by  the  different  focus  distance  motor  settings  used  for  each  band  and  by  chro¬ 
matic  aberration.  In  Constant  Magnification  Focusing  the  model  is  used  to  determine  the 
changes  in  the  focal  length  required  to  compensate  for  the  magnification  changes  resulting 
from  variations  in  the  focussed  distance  of  the  lens. 


Magnification  model 

The  general  form  of  our  relativ<’  magnification  model  is 

M  =  gifdJi) 


where  fd  and  /;  are  the  focal  length  and  focussed  distance  of  the  lens  respectively.  In  our 
model  the  fd  and  /;  values  are  specified  in  motor  units.  The  relative  image  magnification 
between  two  lens  positions  is  found  by  taking  the  ratio  of  the  model  at  those  two  lens 
positions,  ie. 


relative  image  magnification  = 


gjfdiJh) 
gi  fd2  ’  fh ) 


To  mathematical'y  model  the  relative  image  magnification  we  use  a  bivariate  cubic  polyno¬ 
mial  ” 


u  =  aoo  +  floil/  +  ao2’/  +  +  n\ox  +  ai\xy  +  +  0-21  y  +  a^ox^ 

where 

u  is  the  relative  image  magnification 
X  is  the  focus  distance  motor  position 
y  is  the  focal  length  motor  position,  and 

a,j  are  polynomial  coefficients  determined  by  a  cahbration  procedure. 

Calibration  of  the  bivariate  cubic  polynomial  model  is  done  by  making  a  series  of  mag¬ 
nification  measurements  over  a  range  of  focus  distance  and  focal  length  motor  positions  and 
then  using  these  measurements  as  control  points  in  a  least  squares  fitting  procedure.  The 
magnification  measurements  are  made  by  measuring  the  width  of  the  bar  on  the  calibration 
target  shown  in  Figure  .32.  To  calculate  the  width  of  the  bar  the  positions  of  the  white  to 

^Wp  had  initially  tried  using  a  closed-form  optics  mo<iel  but  we  found  that  the  calibration  data  could  be 
more  accurately  followed  by  using  a  bivariate  cubic  polynomial. 
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Figure  32:  Calibration  target  for  relative  image  magnification  model 


to  black  and  the  black  to  white  edges  are  determined  using  the  edge  detection  procedure 
described  in  appendix  A.  The  target’s  width  is  then  calculated  as  the  difference  in  the  tw  - 
edge  positions.  To  avoid  the  effects  of  scanline  timing  jitter  and  pixel  resampling  [1]  width 
measurements  are  made  along  vertical  image  columns.  The  dimensions  of  the  calibration 
target  are  chosen  so  that  the  horizontal  bar  occupies  approximately  67%  of  the  camera's  ver¬ 
tical  field  of  view  at  the  point  of  highest  magnification  [fd>fi)  =  (0,0),  and  appro.ximately 
10%  of  the  vertical  field  of  view  at  the  point  of  lowest  magnification  (fdifi)  =  (3000.3900). 

One  drawback  in  calculating  image  magnification  from  the  width  of  a  single  fixed  cal¬ 
ibration  target  is  the  fact  that  as  the  lens  parameters  are  changed  the  target’s  edges  can 
move  quite  far  across  the  image.  .As  a  result  the  positions  of  the  target’s  edges  and  thus  the 
target's  width  measurement  can  potentially  be  affected  by  local  geometric  distortions  in  the 
image.  We  make  the  assumption  that  the  errors  introduced  by  local  geometric  distortions 
are  small  compared  to  the  changes  in  the  global  image  magnification.  We  also  make  the 
assumption  that  any  biases  caused  by  the  variation  in  the  camera’s  point  spread  function 
across  the  sensor  will  also  be  relatively  small. 

The  final  calibration  data  set  consists  of  target  width  measurements  u  taken  at  focus 
distance  motor  and  focal  length  motor  positions  x  and  y.  For  our  model  we  make  mea¬ 
surements  at  31  foeal  length  positions  and  31  focus  distance  positions  for  a  total  of  96i 
measurement  points.  Focal  length  motor  positions  are  varied  from  0  (/;  %  7.5mm)  to  3900 
( /;  ~  12.5mm).  Focussed  distance  motor  positions  are  varied  from  0  {fd  =  1.2m)  to  3900 
[fi  =  >c). 
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Using  the  pseudo  inverse  procedure  described  in  [18]  the  least  squares  coefficients  of  best 
fit  for  the  model  are 


«00 

6.7052 

Ooi 

= 

-2.8650  X  10'^ 

<102 

4.6365  X  10“'^ 

<l03 

= 

-2.4321  X  10~“ 

<iio 

= 

-5.2330  X  10'** 

ail 

= 

2.4753  X  10'’’ 

ai2 

-3.1141  X  10~’^ 

<I20 

= 

1.6786  X  10~* 

021 

= 

-5.7644  X  10~^2 

<l30 

6.3530  X  10-'^ 

Figure  33  contains  a  plot  of  the  lens  magnification  relative  to  the  position  (0,0).  Figure  34 
contains  a  plot  of  the  difference  between  the  calibrated  model  and  the  calibration  data, 
expressed  as  a  percentage  error  relative  to  the  calibrated  data.  The  majcimum  error  between 
the  model  and  the  calibration  data  is  —3.1%  at  point  (3900,3900),  with  the  mean  and 
standard  deviations  of  the  error  being  0.0016%  and  0.48%  respectively. 
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Figure  33;  Relative  image  magnification  model 


rvM'vc  image  magnificalion 


Figure  34:  Percentage  error  in  relative  image  magnification  model 


percenUge  relative 


Using  the  model 

In  color  imaging  our  objective  is  to  take  red,  green  and  blue  images  with  the  same  relative 
image  magnification,  ie. 


Qrifdri  fir)  ~  Qg(fdgi  fig)  —  Qbifdi,}  fli,) 

With  ACI  fdr^  fdg  and  fd^  are  determined  by  using  an  auto  focusing  routine  or  by  using 
a  calibrated  focussed  distance  model.  Generally  fd^  and  fd^  will  not  be  the  same 
because  of  longitudinal  chromatic  aberration.  Fixing  one  of  the  focal  length  parameters 
(//j,  in  our  implementation),  the  remaining  two  focal  length  parameters  can  be  determined 
by  searching  for  the  settings  that  most  closely  satisfy  the  equality  constraint  on  the  relative 
image  magnification. 

In  range  from  focus  our  objective  is  to  take  a  series  of  images  at  varying  focus  distances 
while  maintaining  the  same  relative  image  magnification,  ie. 


gifd.Jh)  =  gifd-iJh)  =  ....  =  g{fd„,fi„) 


With  CMF  the  focus  distance  parameters  /j,  ...  fd^  are  determined  by  the  application 
performing  the  focusing.  After  fi.xing  one  of  the  focal  length  parameters  (say  //, )  to  establish 
a  base  image  magnification,  the  remaining  focal  length  parameters  are  then  determined  by 
searching  for  the  parameter  values  that  most  closely  satisfy  the  equality  constraint  on  the 
relative  image  magnification.  Effectively  this  causes  the  camera  lens  to  operate  on  an 
isomagnification  contour  on  the  surface  shown  in  Figure  33.  This  can  be  more  clearly  seen 
in  Figure  35  where  the  lens  motor  positions  have  been  plotted  for  one  such  contour. 


39 


focal  length  motor  position 


uncompensated  focusing 
Constant  Magnification  Foe 


Figtire  35:  Constant  Magnification  Focusing  isomagnification  contour 


8  Appendix  C  -  Criterion  function  peak  position  detection 

In  both  range  from  focus  and  autofocus  we  need  to  determine  the  position  of  the  peak  in 
a  set  of  sharpness  criterion  function  measurements  taken  over  a  range  of  focus  distance 
positions.  Typically  the  presence  of  measurement  noise  makes  simple  gradient  searches 
over  the  criterion  function  values  unreliable  and  inaccurate.  One  approach  to  determine 
the  position  of  the  peak  of  the  criterion  function  reliably  and  accurately  is  to  fit  a  model 
to  the  data  and  then  find  the  position  of  the  model’s  peak.  Unfortunately  the  processes 
underlying  the  data  are  scene  dependent  and  generally  very  difficult  to  model.  For  the 
calibration  work  described  in  this  report  we  can  simplify  the  problem  somewhat  by  using 
targets  that  produce  unimodal  criterion  function  responses.  Unfortunately  the  form  of  the 
unimodal  curve  still  varies  with  the  imaging  parameters  and  is  still  difficult  to  determine. 
To  get  around  this  problem  instead  of  trying  to  fit  all  of  the  data  to  a  comple.x  model  we 
fit  a  subset  of  the  data  from  focus  positions  immediately  surrounding  the  peak  to  a  simple 
quadratic  model.  The  position  of  the  peak  in  this  locally  fit  model  is  then  taken  as  the 
position  of  the  criterion  function  peak. 

The  data  and  model 

Our  data  consists  of  criterion  function  measurements  y  taken  at  known  focus  positions  x. 
.\t  each  focus  position  a  number  of  criterion  function  evaluations  are  made  providing  a 
mean  and  standard  deviation  for  the  criterion  function  at  that  focus  position.  So  the  data 
consists  of  N  chosen  focus  positions 


X  =  {ari,X2,-.,i;v} 


having  criterion  function  means 


and  standard  deviations 

a  =  {<Ti,<T2,  ...,a.v} 

To  model  the  relationship  between  x  and  y  we  use  the  second  order  polynomial 

y  =  oi  +  a2X  +  ajx^ 

Fitting  a  quadratic  model  using  least  squares  linear  regression 

To  find  the  coefficients  of  best  fit  for  the  quadratic  model  using  least  squares  linear  regression 
we  need  to  solve  the  following  matrix  equation  (12,  1.5]: 
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we  can  solve  for  the  coefficients 
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ai 
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Finding  Xnax  and  its  variance  cr 


,2 

max 


To  find  the  position  of  the  peak  of  the  fitted  quadratic  curve  we  set  the  first  derivative  of 
the  model  to  zero, 

dy 

^  -  a2  +  2a3X  =  0 
ox 


and  then  directly  solve  for  Xr 


Q2 

2a3 


To  determine  the  variance  of  the  position  of  the  peak  of  the  data  we  first  calculate  the 
Jacobian  of  Xmai  wrt  a: 


J  = 


Ox, 


da 


■  0  ■ 
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da2 
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L  H  J 

From  equation  1  we  also  have  the  linear  transformation  between  a  and  y 

dy 


(3) 


Thus,  from  equations  2  and  3  we  can  define  a  linear  transformation  between  our  calculated 
peak  position  Jmax  y 
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dXr 
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If  we  assume  the  mecisurements  y,  to  be  independent,  with  covariance  matrix  Pyy 
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then  can  be  approximately  determined  using  the  covariance  matrix  and  the  linear 

transformation  defined  in  equation  4 


«=i 

As  j  (and  thus  d)  represents  a  linearization  of  at  Xmax,  the  value  of  will  only  be 

an  approximation. 


Choosing  a  subset  of  data 

In  our  approach  instead  of  trying  to  fit  data  from  the  full  range  of  focus  positions  to  a 
complex  model  of  the  criterion  function,  we  fit  the  data  from  a  smaller  subrange  of  fo¬ 
cus  positions  immediately  surrounding  the  apparent  peak  to  a  simpler  quadratic  function. 
The  assumption  here  is  that  at  least  locally  the  criterion  function  behaves  as  a  symmetric 
unimodal  function  corrupted  by  measurement  noise.  Modeling  this  with  a  quadratic  func¬ 
tion  provides  a  computationally  efficient  means  of  estimating  the  position  of  the  criterion 
function's  peak  and  its  variance. 

Naturally  the  more  data  that  is  used  in  fitting  the  model,  the  more  accurate  our  esti¬ 
mation  of  the  peak’s  position  will  be.  To  determine  just  how  much  data  we  can  use  from 
around  the  peak  we  use  a  chi-squared  figure  of  merit  Q  [12].  Q  gives  a  measure  of  the 
goodness-of-fit  between  a  set  of  data  and  a  model.  The  test  is  defined  as: 


=  1: 

1=1 


g  =  Q(x2|r/)  =  r(^,^) 

where  P  is  the  incomplete  gamma  function  and  1/  is  the  number  of  degrees  of  freedom  (.V  -  .3 
for  our  model).  Q  is  the  probability  that  the  observed  chi-square  will  exceed  the  value 
by  chance,  even  for  a  correct  model.  Values  of  Q  <  0.001  indicate  a  poor  fit  between  the 
model  and  the  data,  with  the  most  probable  reason  for  the  poor  model  fit  being  that  the 
model  is  wrong  for  the  data.  Values  of  g  >  0.001  indicate  a  reasonable  fit  between  the  data 
and  the  model. 

To  determine  the  subset  of  data  to  be  used  in  fitting  the  quadratic  model  we  start  by 
fitting  the  full  data  set  to  the  model  using  the  least  squares  linear  regression  procedure. 
This  gives  us  an  estimated  peak  position  Jmox  and  a  value  for  Q.  If  g  is  >  0.01  we're  done. 
If  g  is  <  0.01  we  drop  the  data  point  that  is  furthest  from  the  estimated  peak  position  and 
repeat  the  process. 

This  data  fitting  approach  is  easily  adapted  to  an  auto  focus  algorithm  by  having  the 
camera  system  replace  the  farthest  data  point  in  the  data  set  by  a  new  measurement  that 
is  taken  at  a  focus  position  that  is  closer  to  the  estimated  peak.  Again  the  approach  is 
continued  until  the  fit  between  the  working  set  of  data  and  the  quadratic  model  is  acceptable. 

In  an  automated  focus  algorithm  the  data  fitting  approach  has  several  advantages  over 
more  conventional  gradient  search  schemes.  For  one,  the  peak  position  is  estimated  from  a 
set  of  measurements  and  their  variances  taken  over  a  range  of  focus  positions,  rather  than 


43 


from  a  sequence  of  decisions  based  on  measurements  made  at  single  focus  positions.  This 
greatly  improves  the  accuracy  and  robustness  of  the  peak  position  detection  in  the  presence 
of  mecisurement  noise.  Another  advantage  of  the  data  fitting  approach  is  the  confidence 
interval  that  it  gives  for  the  estimated  peak  position. 

Peak  detection  example 

Figure  36  shows  the  detected  peak  in  a  set  of  Tenengrad  criterion  function  data  collected 
for  a  fairly  narrow  aperture  (540  motor  units).  For  each  data  point  the  mean  and  1  a  error 
tolerances  are  shown.  The  full  data  set  contains  iOl  points,  of  which  only  29  were  used 
to  fit  the  final  curve.  The  final  figure  of  merit  Q  for  the  curve  was  0.125.  The  estimated 
variance  of  the  peak  position  of  1426  motor  units  was  4  motor  units. 
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